Existing MapReduce implementations only support static parameter configuration at the job level. However, static job configuration faces multiple challenges in areas of performance, scaling, etc. For example, existing static job configuration approaches face difficulties with tuning performance, as well as leveraging the elasticity of cloud infrastructure in real-time.
Additionally, static job configuration approaches can prohibit solving outlier problems, which significantly degrades MapReduce application performance. Further, resource contentions among multiple jobs in a multi-tenant environment are commonly dynamic, and cannot be resolved by static job configuration.
Accordingly, a need exists for techniques that enable dynamic job configuration in MapReduce.